United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



| ATTORNEY DOCKET NO~ CONFIRMATION NO. 



09/974,707 



20350 



10/09/2001 



Martin Wustefeld 



7590 



12/03/2004 

TOWNSEND AND TOWNSEND AND CREW, LLP 
TWO EMBARCADERO CENTER 
EIGHTH FLOOR 

SAN FRANCISCO, CA 94111-3834 



089324-OOOOOOUS 



7870 



EXAMINER 



KRONENTHAL, CRAIG W 



ART UNIT 



PAPER NUMBER 



2623 

DATE MAILED: 12/03/2OO4 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Mciion ournrnary 


Application No. 

09/974,707 


ADDlicantfs) 

WUSTEFELD ET AL 


Examiner 

Craig W Kronenthal 


Art Unit 

2623 





~ The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)|3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) [x] Claim(s) 1-22 is/are pending in the application. 

4a) Of the above claim(s) ____ is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) IEI Claim(s) 1-3,6.7.10,11.13,18,21 and 22 is/are rejected. 

7) [X] Claim(s) 4.5.8.9, 12, 14-17, 19 and 20 is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) d The specification is objected to by the Examiner. 

10)[x] The drawing(s) filed on 09 October 2001 is/are: a)Q accepted or b)[X] objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)[X] Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)KI All b)Q Some * c)Q None of: 

1 - □ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 



Drawings 

1 . The drawings are objected to because the flow diagrams in Figures 1 and 2 do not 
contain text indicating the appropriate names or functionality. Each object of the flow 
diagram that is given a reference number should contain text. Corrected drawing 
sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to 
avoid abandonment of the application. Any amended replacement drawing sheet should 
include all of the figures appearing on the immediate prior version of the sheet, even if 
only one figure is being amended. The figure or figure number of an amended drawing 
should not be labeled as "amended." If a drawing figure is to be canceled, the 
appropriate figure must be removed from the replacement sheet, and where necessary, 
the remaining figures must be renumbered and appropriate changes made to the brief 
description of the several views of the drawings for consistency. Additional replacement 
sheets may be necessary to show the renumbering of the remaining figures. The 
replacement sheet(s) should be labeled "Replacement Sheet" in the page header (as 
per 37 CFR 1 .84(c)) so as not to obstruct any portion of the drawing figures. If the 
changes are not accepted by the examiner, the applicant will be notified and informed of 
any required corrective action in the next Office action. The objection to the drawings 
will not be held in abeyance. 
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Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form 
the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

3. Claims 1 , 7, -10, 1 1 , and 18 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Nagasaka et al. (PN 6,157,744). (hereinafter Nagasaka) 

Regarding Claims 1 and 18: Nagasaka discloses a method for the detection of an 
object moving in the monitored region of a camera, wherein 

• An actual image of the monitored region is recorded by the camera (Fig. 1,11) 
(col. 4 lines 8-9). The interface (Fig. 1 , 8) which enters digital image data may be 
a camera. 

• At least one actual measured value. is derived from the actually recorded image 
which provides information on differences between at least two different image 
regions and which is invariant with respect to image displacements, image 
rotations and/or image size changes (col. 4 lines 34-39). The actual measured 
value in Nagasaka is represented by the data of a color histogram. The color 
histogram provides information on color differences between multiple regions of 
an image. Each object in an image is represented by a plurality of pixels with 
similar color values. Peaks on a color histogram show differences between 
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objects and their background. Also color histograms are invariant with respect to 
image displacements and image rotations. 

• This actual measured value is compared with a corresponding reference value 
derived from a stored reference image recorded by the camera (col. 4 lines 50- 
53). The CPU (Fig. 1 , 7) calculates the difference between a color histogram 
representing a current frame and the color histogram of a previous or reference 
frame. 

• An object recognition reaction is triggered on a pre-set deviation of the actual 
measured value from the reference value (col. 8 lines 40-44). The step 614 (Fig. 
6) is triggered by step 612 (Fig. 6) when a threshold is exceeded. The object 
recognition reaction produced by step 614 is not specified, as it could be 
programmed to be a number of desired results. 

Regarding Claim 7: Nagasaka discloses a method in accordance with claim 1 , 
characterized in that a reaction is triggered when the comparison of the actual 
measured value with the reference value provides a comparison value which lies 
outside a pre-set tolerance range (col. 8 lines 40-44). The reaction done by step 614 is 
triggered by step 612 which compares the differences between the actual color 
histogram and a reference color histogram and determines if this difference is above a 
predetermined threshold. 
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Regarding Claim 10: Nagasaka discloses a method in accordance with claim 1 , 
characterized in that a plurality of actual measured values of the same or of a different 
type is derived from the actual image information or structure information and is 
compared with corresponding stored reference values derived from the image 
information or the structure information of the reference image (col. 4 lines 34-39). The 
actual measured values in Nagasaka are contained in a color histogram. The color 
histogram shows the frequency of each color in the image. The color histogram 
provides information on color differences between multiple regions of an image. Each 
object in an image is represented by a plurality of pixels with similar color values. 
Peaks on a color histogram show differences between objects and their background. 
The CPU (Fig. 1 , 7) calculates the difference between a color histogram representing a 
current frame and the color histogram of a previous or reference frame. 

Regarding Claim 1 1 : Nagasaka discloses a method in accordance with claim 1 , 
characterized in that the measured value(s) or reference value(s) include the following 
information: 

• Color difference between two different image regions or structure regions or 
between a reference point and an image region or a structure region (col. 4 lines 
34-39). The actual measured value in Nagasaka is represented by the data of a 
color histogram. The color histogram provides information on color differences 
between multiple regions of an image. Each object in an image is represented 
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by a plurality of pixels with similar color values. Peaks on a color histogram show 
differences between objects and their background. 

4. Claims 1 and 6 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Kawabata et al. (PN 4,783,833). (hereinafter Kawabata) 

Kawabata also discloses a method for the detection of an object moving in the 
monitored region of a camera, wherein 

• An actual image of the monitored region is recorded by the camera (Fig. 1, 101) 
(col. 2 lines 50-57). The image pick-up element (101) may be a camera. 

• At least one actual measured value is derived from the actually recorded image 
which provides information on differences between at least two different image 
regions and which is invariant with respect to image displacements, image 
rotations and/or image size changes (col. 3 lines 4-6). The actual measured 
values in Kawabata are the points of the edge enhancement produced by the 
feature extracting unit (Fig. 1, 103). The edge enhancement provides information 
on distance differences between multiple regions of an image. For instance, in 
figure 2, edge enhancement (201 ) shows the distance the outline of the person is 
from the outline of the house. Also edge enhancement images are invariant with 
respect to image displacements and image rotations. 

• This actual measured value is compared with a corresponding reference value 
derived from a stored reference image recorded by the camera (col. 5 lines 16- 
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25). The comparison and background edge image renewal unit (Fig. 1 , 107) 
calculates the difference between an input edge enhancement image and a 
background edge enhancement image. 
• An object recognition reaction is triggered on a pre-set deviation of the actual 
measured value from the reference value (col. 5 lines 56-59). The reaction is the 
outputting of a signal 1j (Fig. 1 ) after thresholding with a pre-set deviation, T (col. 
5 lines 43-45). 

Regarding Claim 6: Kawabata discloses a method in accordance with claim 1 , 
characterized in that an image of an object-free monitored region is used as the 
reference image (Fig. 2, 204-206) (col. 3 lines 65-66). The reference images (204-206) 
show just the house without the corresponding objects in motion, namely the person, 
cat, and bird as shown in the input images (201-203). 



Claim Rejections • 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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6. Claims 2, 3, and 21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nagasaka et al. in view of Kawachi et al (PN 6,285,787). ^hereinafter Kawachi) 

Regarding Claim 2: Nagasaka discloses the method in accordance with claim 1 , but 
does not disclose the use of a structure. Kawachi discloses an image processing 
method, characterized in that a structure (slit plate, Fig. 2, 27), which is a component of 
the image, is superimposed (col. 4 lines 41-43) on the reference image and the actually 
recorded image (col. 6 lines 16-22); and in that the reference value and a measured 
value are gained from the corresponding structure information (col. 6 line 65 - col. 7 line 
3). The slit plate (27) is placed in front of a light source (Fig. 1 , 25) causing structures to 
be projected or superimposed on a field of view (measurement surface, Fig. 1,11). A 
reference image (model image) is captured with the projected structure (27), as is an 
input image when an object (Fig. 1, 1) appears (col. 6 lines 46-48). The structure (27) 
changes the gray values of each image in those regions in which it is superimposed. 
The resulting image data are then used as shading data for comparison. It would be 
obvious to one of ordinary skill in the art to modify Nagasaka with the teachings of 
Kawachi because both references teach the monitoring of a region of interest. 
Furthermore, one would be motivated to use projected structures with Nagasaka's 
method to focus the images and judge whether the image is good. This would help 
minimize erroneous detection resulting from instantaneous disturbances as is desired 
by Nagasaka. 
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Regarding Claims 3 and 21 : Nagasaka as modified by Kawachi discloses the method in 
accordance with claim 2. Furthermore Kawachi discloses that the structure (slit plate, 
Fig. 2, 27) in the monitored region (measurement surface, Fig. 1, 11) is generated 
during the recording of the reference image and of the actual images of the monitored 
region by means of a projection device and/or by means of a moved light beam (col. 4 
lines 41-43). The projection device is the combination of the light source (Fig. 1, 25) 
and the slit plate (27). 

7. Claims 13 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Kawabata in view of Katayama et al. (PN 6,640,004). (hereinafter Katayama) 

Regarding Claims 13 and 22: Kawabata discloses a method in accordance with claim 
1 , but does not disclose the luminating of its monitored region. However Katayama 
does disclose an image sensing apparatus and process characterized in that the 
monitored region is illuminated by means of at least one light source (illumination unit, 
Fig. 2, 200) during the recording of the reference image and of the actual images of the 
monitored region (col. 6 lines 65-67). It would be obvious to one of ordinary skill in the 
art to modify Kawabata with Katayama since both references teach the comparing of 
input images to background images for the purpose of object detection. Furthermore, 
one would be motivated to make this modification to illuminate the monitored region so 
the camera may capture a clear image of all objects within the monitored region under 



Application/Control Number: 09/974,707 Page 10 

Art Unit: 2623 

all conditions. Often object detection is utilized in areas that do not receive sufficient 
sunlight or artificial lighting for a video camera. 



Allowable Subject Matter 

8. Claims 4, 5, 8, 9, 12, 14-17, 19, and 20 are objected to as being dependent upon a 
rejected base claim, but would be allowable if rewritten in independent form including all 
of the limitations of the base claim and any intervening claims. 

Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

• Courtney (PN 5,969,755) is cited for teaching video surveillance and a 
method of automatic object detection including the use of motion 
segmentation. 

• Crabtree et al. (PN 6,295,367) is cited for teaching video surveillance and a 
method for tracking a moving object in a specified viewing area utilizing 
region segmentation and background image generation. 

• Ito et al. (PN 6,088,468) is cited for teaching video surveillance utilizing image 
differencing and binarization to detect an object. 

• Matsuga et al. (PN 6,167,167) is cited for teaching image extraction utilizing a 
mask and difference imaging. 
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• Munno et al. ("Automatic Video Image Moving Target Detection for Wide Area 
Surveillance") is cited for teaching image filtering and segmentation for object 
detection. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Craig W Kronenthal whose telephone number is (703) 
305-8696. The examiner can normally be reached on 8:00 am - 5:00 pm / Mon. - FrL 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (703) 306-6604. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



CWK . MEt-MDAD DASTOURI 

1 1 /05/04 PRIMARY EXAMINER 



